Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximation
نویسندگان
چکیده
Recently, deep reinforcement learning (RL) has achieved remarkable empirical success by integrating neural networks into RL frameworks. However, these algorithms often require a large number of training samples and admit little theoretical understanding. To mitigate issues, we propose theoretically principled nearest neighbor (NN) function approximator that can replace the value in methods. Inspired human similarity judgments, NN estimates action values using rollouts on past observations provably obtain small regret bound depends only intrinsic complexity environment. We present (1) Nearest Neighbor Actor-Critic (NNAC), an online policy gradient algorithm demonstrates practicality combining approximation with RL, (2) plug-and-play update module aids existing Experiments classical control MuJoCo locomotion tasks show NN-accelerated agents achieve higher sample efficiency stability than baseline agents. Based its benefits, believe be further applied to other complex domains speed-up learning.
منابع مشابه
Symmetry Detection and Exploitation for Function Approximation in Deep RL
With recent advances in the use of deep networks for complex reinforcement learning (RL) tasks which require large amounts of training data, ensuring sample efficiency has become an important problem. In this work we introduce a novel method to detect environment symmetries using reward trails observed during episodic experience. Next we provide a framework to incorporate the discovered symmetr...
متن کاملAcceleration of Binning Nearest Neighbor Methods
A new solution method to the Nearest Neighbour Problem is presented. The method is based upon the triangle inequality and works well for small point sets, where traditional solutions are particularly ineffective. Its performance is characterized experimentally and compared with k-d tree and Elias approaches. A hybrid approach is proposed wherein the triangle inequality method is applied to the ...
متن کاملFractal Image Compression via Nearest Neighbor Search
In fractal image compression the encoding step is computationally expensive. A large number of sequential searches through a list of domains (portions of the image) are carried out while trying to find best matches for other image portions called ranges. Our theory developed here shows that this basic procedure of fractal image compression is equivalent to multi-dimensional nearest neighbor sea...
متن کاملNearest neighbor search through function minimization
This paper describes a solution to the nearest neighbor problem. The proposed algorithm, which makes use of the triangle inequality property, is considered from a function minimization perspective. The distance function is regularized through the computation of distance to a reference point; an initial starting point is rapidly found, and used in an iterative refinement using search over a sort...
متن کاملAn Efficient Approximation-elimination Algorithm for Fast Nearest-neighbor Search
In this paper, we present an efficient algorithm for fast nearest-neighbour search in multidimensional space under a so called approximation-elimination framework. The algorithm is based on a new approximation procedure which selects codevectors for distance computation in the close proximity of the test vector and eliminates codevectors using the triangle inequality based elimination. The algo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2021
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v35i11.17151